Journal of the American Medical Informatics Association

Objective: To estimate Long COVID undercoding in electronic health records (EHRs) against federal survey prevalence, and to evaluate a predictor of later coding that anyone can rebuild from counts a research enclave releases. Materials and Methods: Among 23,790 All of Us adults with COVID-19, we compared coded shares with Household Pulse Survey prevalence. Among 23,300 adults uncoded at day 90, we predicted a first U09.9 code by day 455. We fitted naive Bayes and linear discriminant models outside the enclave from released counts and compared them with an elastic net fitted inside to the same patients and features. Tests were a random 20% of patients (4,680, 50 coded) and 18 held-out EHR sites (5,760, 60 coded). Results: Under survey-comparability assumptions, about 1 in 9 adults who ever had Long COVID carried its code, and the women-to-men coding ratio was 0.79 (95% sensitivity interval, 0.65-0.95). The released models' areas under the receiver operating characteristic curve (AUROCs) were 0.68 (95% CI, 0.60-0.75) on the patient test and 0.64 (0.55-0.72) at held-out sites, 0.049 (0.001-0.104) above the elastic net. On the patient test, the elastic net had a higher area under the precision-recall curve (difference 0.062, 0.003-0.122). Released probabilities overestimated coding risk at held-out sites. Discussion: Fitting from released counts did not consistently lower discrimination. Coded outcomes inform disease discrimination only under stated coding assumptions. Conclusion: Long COVID is substantially and unevenly undercoded, and a model rebuilt from public counts predicts its later coding with moderate discrimination.

Background. Research data enclaves such as the All of Us Research Program do not allow participant-level data, or models fitted to them, to leave, which makes it hard to deliver case-finding models, which flag patients with possibly uncoded conditions for clinician review, to patients and clinicians. Objectives. We describe and technically evaluate a system that fits models outside an enclave from released summary counts and delivers them through standard health information technology interfaces. Methods. We exported dissemination-compliant summary counts from the All of Us Researcher Workbench and, outside the enclave, fitted versioned models for Long COVID, chronic fatigue, fibromyalgia, endometriosis, Lyme disease, lupus, migraine, depression, and anxiety. A model contract (a model file and a written specification), a Python reference implementation, and synthetic Fast Healthcare Interoperability Resources (FHIR) test patients define the expected outputs. One JavaScript scorer runs in a patient demonstration, a SMART on FHIR clinician application, and a CDS Hooks service. An atlas displays aggregate results. We evaluated the system on synthetic records with automated tests, cross-implementation output comparison, launches through the public SMART sandbox, and a counterexample review of the scoring and display rules. Results. The deployed Long COVID model had an area under the receiver operating characteristic curve of 0.67 in a random validation split. The eight self-report models reached 0.61 to 0.82 in identifying self-reporters without a code. The JavaScript scorer reproduced all 162 reference outputs exactly, and all 437 automated tests passed. A counterexample review found 5 classes of failure, including scoring ineligible patients and treating unavailable data as absent, all fixed and added as tests. Conclusions. Models fitted from released summary counts can be delivered through FHIR, SMART on FHIR, and CDS Hooks with reproducible scoring on synthetic records. Real-record interoperability, usability, and prospective clinical performance remain to be evaluated.

Clinical records routinely document medical interventions but often omit the reasoning connecting patient-specific evidence to medical action. This limits the decision-relevant supervision available for training models for personalized care. Here we evaluate a data-augmentation approach that makes selected clinical decision relationships explicit in an 8-billion-parameter Qwen3 model, ChronIQ-DRx2. Using the same training cases and model architecture, expert-guided logic reconstruction was applied across four chronic diseases. On the held-out evaluation, the enhanced model had the highest plotted end-to-end task-completion rate among evaluated systems and higher plotted scores across diagnostic, medication and follow-up measures. The serious clinical error rates shown were 16.7% for the baseline and 12.5% for the enhanced model. In the human-AI evaluation, physician accuracy was 87.40% alone and 88.20% with AI assistance, while mean completion time was 0.959 and 0.780 min, respectively; medical-student accuracy was 62.71% and 74.80%, and mean completion time was 0.210 and 0.195 min. These results are descriptive point estimates; inferential comparisons are not reported here. Structured outputs may make recommendations easier to review, but rationale faithfulness, generalizability and effects on clinical safety require further evaluation.

Large language models (LLMs) are increasingly consulted for health advice, but they can produce fluent statements that are false or unsupported, known as hallucinations, and this risk is poorly characterised in low-resource languages such as Bangla. We measured hallucinations of locally deployable open-weight LLMs in Bangla diabetes question answering and tested whether retrieval-augmented generation (RAG) grounded in Bangladeshs National Guideline on Diabetes Mellitus reduces them. Forty patient questions spanning six clinical sub-domains were each answered twice, under zero-shot prompting, by Qwen3.5-9B, TigerLLM-9B, Qwen3.6-35B-A3B and Gemma-4-26B-A4B, and by BanglaDiaRAG, which adds hybrid dense (BGE-M3) and sparse (BM25) retrieval from a physician-reviewed Bangla guideline knowledge base to Qwen3.5-9B. Two physicians rated the 400 de-identified responses by consensus for medical accuracy, completeness, safety, readability and reliability on a 0-1 scale, and responses with accuracy below 0.70 were classified as hallucinated. Hallucination rates were 2.5% for Gemma-4-26B-A4B, 3.8% for TigerLLM-9B, 22.5% for Qwen3.6-35B-A3B and 77.5% for Qwen3.5-9B. Guideline grounding lowered the rate of Qwen3.5-9B to 11.3% (difference -66.3 percentage points; 95% CI -76.3 to -55.0) and raised every criterion by 0.24 to 0.38 points (Holm-adjusted p < 0.001). BanglaDiaRAG outscored Qwen3.6-35B-A3B on all five criteria and matched or exceeded TigerLLM-9B, but it remained below Gemma-4-26B-A4B in safety, readability and reliability, and its residual errors were specific, confident and potentially harmful. Guideline grounding can turn a weak general-purpose model into a much safer Bangla diabetes assistant, but the choice of base model mattered as much as retrieval and no configuration was free of hallucination, so clinical oversight remains essential before patient-facing use.

Background: Ambient artificial-intelligence (AI) scribes are being deployed beyond medicine into allied-health practice, but comparative evidence mainly concerns physician documentation. Objective: To compare quality and transcript-grounded clinical-risk error burden of AI-generated and allied health professional notes from identical simulated consultations. Methods: Paired simulation across dietetics, physiotherapy, occupational therapy and speech and language therapy (five consultations each; 20 pairs; 40 notes). Heidi generated an unedited first-draft note and a professional documented each encounter. Professionals rated PDQI-9 quality. Professional and automated review identified unsupported statements and omissions; two language models graded clinical risk. Co-primary outcomes were PDQI-9 score and Critical+High error density. Results: Mean PDQI-9 scores were 38.45 and 35.05 for AI and professional notes (paired difference +3.40, 95% CI +0.01 to +6.79; Wilcoxon p=0.042; dz=0.47). The Bayesian estimate was +3.47 (94% equal-tailed credible interval +0.66 to +6.30; P[AI>professional]=0.989). Automated-reviewer all-error density was 3.35 vs 8.55 per note (professional-minus-AI difference 5.20, 95% CI 3.85 to 6.55); omission density was 1.45 vs 5.20 (difference 3.75, 2.80 to 4.65). Critical+High density was 0.35 vs 0.75 (difference 0.40, -0.10 to 1.00). The pooled quality result was driven principally by dietetics. Conclusions: AI-generated notes achieved modestly higher pooled quality and fewer transcript-grounded errors, chiefly omissions; serious-error evidence was inconclusive. Small discipline samples and incomplete ground truth preclude safety or discipline-wide superiority claims.

Importance: Conditions that are hard to diagnose, several more common in women, may be missing from electronic health records (EHRs), hiding patients from research cohorts and case finding. Objective: To measure how often the EHR contains a diagnosis code for chronic conditions that participants report, and to compare this with national survey benchmarks. Design, Setting, and Participants: This cross-sectional study used the All of Us Research Program Registered Tier Dataset v9. Participants were adults with EHR visits in both 12-month periods from July 2021 to June 2023 who completed a health history survey. We analyzed the data in October 2026. Exposures: Self-reported clinician diagnosis of 10 conditions with ongoing visits or treatment, as well as sex at birth, race and ethnicity, age, and years of EHR records. Main Outcomes and Measures: Share of participants reporting current care who had a qualifying diagnosis code, alone and relative to hypertension and type 2 diabetes, and the ratio of age-standardized coded shares to national survey prevalence. Results: Of 161,480 respondents, 114,610 (73,870 female [64.5%], median [IQR] age 63 [50-72] years) had at least 5 years of EHR records before the survey. Among those reporting current care, 84.6% (95% CI, 84.2%-85.0%) with hypertension and 87.0% (95% CI, 86.5%-87.6%) with type 2 diabetes had a code, compared with 41.3% (95% CI, 38.9%-43.8%) for endometriosis, 39.4% (95% CI, 34.3%-44.8%) for Lyme disease, and 24.4% (95% CI, 23.2%-25.6%) for chronic fatigue. Counting ICD-9-CM and other mapped codes, 55.3%, 51.5%, and 28.3%, respectively, had any qualifying record. Women had lower coded shares than men for hypertension, type 2 diabetes, obesity, depression, and anxiety (crude ratios 0.93-0.96, within-health-system ratios 0.98-1.02) and higher shares for migraine and fibromyalgia (within health systems, 1.11 [95% CI, 1.07-1.15] and 1.41 [95% CI, 1.29-1.57]). Age-standardized coded shares among active patients were 1.1 to 2.4 times national prevalence of the same self-reported diagnoses. Conclusions and Relevance: In this cross-sectional study, fewer than half of participants reporting ongoing care for chronic fatigue, Lyme disease, or endometriosis had a qualifying ICD-10-CM code. Linking each participant's report to their own record measured this gap directly.

Awareness of comorbidities in people with dementia is vital context for dementia research and clinical management. Natural Language Processing (NLP) applied to free text in the electronic health record (EHR) may help ascertain comorbidity presence. Our aim was to assess the validity of the MedCAT Comorbidity NLP app in a cohort of people with dementia. We used text from the EHR of South London and Maudsley NHS Mental Health Trust via Clinical Records Interactive Search (CRIS), linked to structured data from local primary care (Lambeth DataNet) and hospital admission data in a cohort of 3,543 people with dementia. We evaluated MedCAT Comorbidity, examining internal validity via manual review of text and external validity by cross-validation against linked primary care and hospital data; and comparison of the predictive value of comorbidity measures for mortality. Internal precision was highest for atrial fibrillation (100%) and lowest for diabetes (78%). Cross-validation with linked data varied substantially across conditions, with precision from 96% to 59% and recall 77% to 4%. The median number of comorbidities identified was two using any single source, or three when combined. MedCAT Comorbidity demonstrated predictive ability for mortality (HR 1.11 per condition), although the model fit was better using comorbidity from hospital admission data. In people with dementia, the MedCAT Comorbidity app showed high precision for most conditions during internal validation. The variability of precision and recall on external validity assessment partly reflects incomplete documentation. Results from the MedCAT comorbidity app can assist researchers in understanding comorbidity of cohorts, particularly when primary care or hospital data is not available. Our findings highlight the need to develop quality assurance protocols that include internal and external validity checks before deploying general data extraction apps for specific clinical subpopulations.

Objective: To determine how cohort versus nested case-control (NCC) design affects EHR-based prediction of a rare outcome. Materials and Methods: We compared two cohort and five NCC designs for predicting early-onset gastrointestinal cancer using L1-regularized logistic regression (LASSO), XGBoost, and multilayer perceptron (MLP). Models underwent nested five-fold cross-validation with model-specific hyperparameter tuning and temporal evaluation in a held-out prospective cohort representing intended deployment. Model discrimination was assessed using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). Model calibration was assessed using Brier scores and calibration plots before and after recalibration with isotonic regression. Feature-ranking similarity was quantified using rank-biased overlap. Results: The cohort development datasets included 256,854-415,529 patients, with 2-yr cumulative incidence of 0.12%-0.17%, compared with approximately 9% in the 1:10 matched NCC datasets. NCC designs generally produced higher internal discrimination, but temporal performance varied by algorithm and index-date definition. Cohort designs achieved the highest temporal AUPRC for LASSO (0.068), whereas diagnosis-anchored NCC achieved the highest temporal AUPRC for XGBoost (0.086) and MLP (0.070). Temporal AUPRC generally decreased as the required lead time before diagnosis increased. Raw predicted probabilities substantially overestimated observed risks in the temporal cohort. Isotonic recalibration improved probability scaling while largely preserving discrimination, although residual miscalibration remained. Cohort-based feature rankings were more consistent across algorithms, whereas NCC-based rankings depended more strongly on algorithm and index-date anchoring. Conclusion: Study design should be selected and evaluated according to the intended clinical prediction task and deployment setting.

Personalized medicine aims to tailor treatments to individual patients, but its implementation is limited by scarce clinical resources and heterogeneous treatment effects. Electronic health records now offer longitudinal data that can support data-driven therapy selection. We focus on SGLT2 inhibitors, medications with broad metabolic and cardiovascular benefits, whose clinical impact varies significantly based on individual patient profiles. We propose a counterfactual Structural Causal Model (SCM) to estimate individual treatment effects (ITE) of SGLT2 inhibitors from observational data. The model assumes counterfactual stability and uses a Gumbel-Max sampling mechanism to capture stochastic treatment assignment while enabling differentiable inference. Optimal treatments are defined by maximizing the counterfactual value of future diagnostic outcomes. We evaluate the Counterfactual Gumbel-Max SCM using a doubly robust value estimator and show that, across diagnostic categories, it outperforms real-world prescriptions in predicted treatment value, highlighting subpopulations where SGLT2 inhibitors yield superior outcomes. By integrating causal inference with electronic health data, this framework provides individualized counterfactual reasoning about SGLT2 treatment effects and can support clinicians in identifying optimal therapies for type 2 diabetes and comorbid conditions.

Background and Objective: The volume of patient portal messages has grown substantially and artificial intelligence (AI) can support clinical staff and improve the timeliness of patient care. Our objective is to test if an AI tool for managing the electronic inbox In-Basket reduces response times to patient portal messages in neurology. Methods: We conducted an implementation study from January to May 2026 in the neurology department at two clinical sites within a regional medical system. A single AI tool was implemented to auto-categorize and auto-route all patient messages received in the electronic inbox. The tool also generates optional draft replies to messages. Primary process measures were the median weekly time (hours) from message creation to the first action and to closure. Impact was analyzed using a controlled interrupted time-series (ITS) design, comparing message-volume-matched control divisions. Results: Among 24,142 messages analyzed, pre-intervention median weekly time to first action was 11.3 hours [IQR: 5.6, 15.1] (Site A) and 16.1 hours [IQR: 13.8, 19.0] (Site B), falling post-intervention to 1.0 hour [IQR: 0, 3.1] (Site A) and 0.6 hours [IQR: 0.2, 2.4] (Site B). Controlled interrupted time-series analysis showed a significant immediate differential level change in median weekly hours to first action at Site A (-92.5%, 95% CI: -97.5% to -78.0%), and a significant downward trend at Site B (-43.5%, 95% CI: -65.2% to -8.2%). Changes in time to closure were directionally favorable but not statistically significant at either site. There was low use of the generated draft replies, only 0.5% at Site A and 3% at Site B. Discussion: Implementation of AI-assisted message categorization and routing was associated with a substantially reduced initial action time for patient In-Basket messages in Epilepsy. The utilization of AI-generated draft-replies to messages was low. These findings support the feasibility and effectiveness of AI-augmented workflows to improve inbox management.

The explosive growth of ambient AI scribes in healthcare has focused almost exclusively on relieving clinician documentation burdens and enhancing clinician efficiency and reducing burnout. While most research has focused on the clinician experience, these same types of tools, such as a patient held AI Personal Health Assistant, can be put in the hands of patients to facilitate communication with healthcare providers, to provide greater autonomy and activation for patients in managing their own healthcare, and to facilitate the task of receiving and retaining complex new information. However, to ensure these tools serve as more than just administrative aids, we partnered with a large technology company to conceptually codesign how such a tool might be studied in real world settings in a way that reflected outcomes of greatest importance to patients and families. We used the Nominal Group Technique and engaged thirty three patients and caregivers and ten multidisciplinary clinicians in structured virtual sessions to identify the outcomes on which subsequent studies should hinge. Our findings reveal a significant misalignment between current industry trends and patient priorities. While the market emphasizes the scribe function for billing, stakeholders overwhelmingly prioritized the AIPHAs role in longitudinal care management and post visit coordination. The most critical outcome identified was the impact on communication and respect, suggesting that patients view AI as a mechanism to improve the human to human dynamic and ensure a shared understanding of the care plan. Our findings further highlighted that the primary hurdle for the use of patient held AI is not technical utility but clinician trust and the need for formal consent processes. Importantly, as this technology is new and integration into the clinical experience is still being explored, patients cannot assume they will have access to the transcript of their own encounter when that record is created and held by the health system. This gap highlights the potential value of a patient held tool, preserving discussions from their visit and follow up steps in clearer detail than they may receive in their clinical note or traditional visit summary. From a system perspective, this transition requires a strategic shift from measuring workflow focused efficiency to measuring patient side advocacy and engagement. By centering the evaluation of AI on these patient defined metrics and fundamentally designing subsequent AI investigations using a partnership model, health systems can move beyond simple automation toward a model of care delivery that empowers patients to navigate complex clinical journeys with greater autonomy and clarity.

Biomedical artificial intelligence (AI) increasingly relies on heterogeneous data sources, yet differences in data structure, terminology, evidence quality, and provenance complicate their integration into reliable knowledge resources. Here, we develop an evidence-aware framework for processing and integrating heterogeneous pharmacovigilance data from FDA Adverse Event Reporting System (FAERS) reports, biomedical literature, clinical trials, and electronic health records in Mass General Brigham. The framework combines large language model (LLM)-based information extraction, ontology-driven semantic normalization, cross-source evidence assessment, statistical and machine learning-based prioritization, and ingredient-based adverse drug event (ADE) relationship inference to transform heterogeneous evidence into a structured and evidence-based knowledge resource. The constructed knowledge system contains 11,588 drug nodes, 44,843 ingredient nodes, 3,275 ADE nodes, 993 indication nodes, and more than 2.47 million relationships, with each drug-ADE relationship assigned a confidence score. We further developed a drug-level web interface for accessing the knowledge resource and validated its utility in downstream AI tasks. Specifically, incorporating the developed knowledge resource improved the comprehensiveness, relevance, and overall quality of LLM-based drug-specific ADE synthesis and improved the precision and F1 score of clinical ADE relation extraction. These results demonstrate that AI-assisted processing combined with evidence-aware curation can transform heterogeneous biomedical evidence into reusable, AI-ready knowledge with explicit provenance and evidence strength information. The framework provides a general strategy for integrating heterogeneous biomedical data to support reproducible knowledge construction and downstream biomedical AI applications.

Background: Hospital readmission is a common and costly outcome in patients with chronic obstructive pulmonary disease (COPD). Multimodal deep learning models combining structured electronic health record (EHR) data with physiological, imaging, and textual data have been proposed to improve its prediction, but whether such fusion outperforms EHR data alone, once temporal-leakage safeguards are applied, remains unclear. Methods: Using a stratified MIMIC-IV cohort of COPD patients enriched to about 50% readmission (natural prevalence 18.1%; n=1,279; 895/192/192 training/validation/test), we predicted 30-day, all-cause readmission from four frozen, pre-trained encoders (electrocardiogram, chest radiograph, clinical notes, structured EHR). We audited and corrected temporal leakage in the admission-linkage pipeline of each modality, then compared three fusion architectures (naive concatenation, scalar gated fusion, EHR-anchored residual fusion) against unimodal baselines and an EHR-alone reference, using bootstrap significance testing, a missingness stress test, and calibration analysis with and without post-hoc recalibration. Results: The audit corrected three previously undetected temporal-alignment issues, affecting 5-9% of patients for clinical notes, 29-43% for chest radiographs, and 53-58% for ECG. After correction, every modality except EHR showed near-chance discrimination alone (ROC-AUC 0.46-0.53), confirmed with a non-linear adapter probe. No fusion architecture significantly outperformed the EHR-alone baseline (ROC-AUC 0.63); removing auxiliary modalities improved rather than degraded fusion performance under stress testing. The raw Brier score was lower for all three fusion architectures than for EHR-alone (about 0.24 versus 0.32), but this reflected overconfidence of the EHR-alone probe (calibration slope 0.12); after logistic recalibration fitted on the validation set, Brier scores were indistinguishable (EHR-alone 0.239; fusion 0.240-0.244; all paired bootstrap 95% CIs included zero). Conclusions: Structured EHR data alone matched the discrimination and, after recalibration, the calibration of more complex multimodal fusion for 30-day COPD readmission prediction. These findings highlight the importance of rigorous temporal-leakage auditing in multimodal clinical prediction research and suggest that reformulating this task around temporal proximity to the outcome, rather than refining the fusion mechanism further, may be more productive for future work.

Background: Conventional discrimination metrics, such as the area under the receiver operating characteristic curve (AUROC), do not characterize the composition of outcomes among patients flagged as high risk by clinical prediction models. This limitation is particularly relevant in critical care, where competing deaths and incomplete follow-up can complicate the interpretation of prediction alerts. We developed the Balanced Risk Integrity Index (BRII), a stratum-level metric defined as the proportion of target events among target events, competing deaths, and early-censored observations in a flagged high-risk stratum. BRII is algebraically related to positive predictive value (PPV) and is reported alongside its component decomposition to characterize the outcome composition of flagged patients. Methods: We evaluated BRII using seven acute kidney injury (AKI) prediction models developed in MIMIC-IV and externally validated in MIMIC-III and eICU-CRD (116,010 patients in total). Additional simulations examined the effects of competing deaths and early censoring on BRII and conventional model performance metrics. Results: For 48-hour AKI prediction in the top 10% risk stratum, BRII ranged from 0.90 to 0.96 in MIMIC-III, compared with less than 0.26 across all models in eICU-CRD, where 61.5% of evaluated patients were censored before the prediction horizon. In the eICU-CRD top 10% strata for the same target, 37.0% to 50.2% of patients were censored early, 12.2% to 15.5% had AKI, and 3.4% to 8.1% died before AKI. BRII and PPV ordered at least one model pair differently in 34 of 48 cohort-outcome-top-k groups. Under simulated random censoring for 48-hour AKI prediction, BRII declined by up to 0.80 while binary-label AUROC changed by less than 0.02. Conclusions: BRII provided complementary information on high-risk stratum composition, yielding different model rankings from PPV and revealing substantial differences across external validation cohorts. Interpreted alongside PPV and its component decomposition, BRII offers a standardized approach to evaluating clinical prediction alerts across models and observation environments.

High attrition rates across clinical drug development impose immense clinical and financial burdens, yet prospectively forecasting whether an individual trial will achieve its primary endpoint remains challenging. Here we present a machine-learning system that predicts clinical trial success across phases I-IV, independently modeling studies with efficacy versus safety/toxicity primary endpoints. To prevent lookahead bias and outcome leakage, models are trained on registration-time features, including target and mechanism-of-action precedent, drug modality and novelty, trial design and operational characteristics, and frozen literature-derived target-choice embeddings. Each prediction yields a continuous Predictive Trial Index (PTI) between 0 and 1 reflecting success probability. On a test set of trials initiated in 2021 or later, the efficacy model achieved an area under the receiver operating characteristic curve (AUROC) of 0.828 (95% CI 0.812-0.845; n = 2,347) and the safety/toxicity model reached 0.809 (95% CI 0.787-0.831; n = 1,485). These models are embedded in inClinico Agent, an agentic AI system integrating frontier large language models (LLMs) with deterministic scoring engines, retrieval tools and specialized clinical skills. We also introduce ClinBench, a prospective benchmark of 2,040 ongoing trials evaluated by inClinico Agent and five frontier LLMs, establishing a leakage-free foundation for tracking prospective predictive accuracy as trials reach completion.

INTRODUCTION: The digital clock drawing test (dCDT) is a cognitive screening tool capturing high-resolution pen-movement data. Existing approaches for classifying cognitive impairment from dCDTs rely on selected features or image-based methods. We propose representing dCDTs as structural networks to classify cognitive impairment and characterize spatial organization in clock drawings. METHODS: Weighted networks were constructed from 3,362 Framingham Heart Study participants, with and without cognitive impairment. Network representations were generated separately for command and copy conditions, with nodes representing pre-defined clock components (digits, hands, clock face) and edges capturing pairwise spatial relationships between components. Random forest models and graph convolutional networks were trained to predict cognitive impairment from the networks. Models were benchmarked against demographics-only and established summary feature baselines. RESULTS: Network-based models achieved AUC values comparable to the established summary feature baseline across command-only and copy-only conditions. Differences in spatial organization between the cognitively intact and impaired groups were greatest in the upper-right clock region, involving digits 1 and 2, the clock hands, and the surrounding clock face. Differences between the command and copy networks were larger among cognitively impaired participants. DISCUSSION: The proposed network representations provide an objective, scalable, and flexible framework for representing clock drawings for use in classification models. dCDT networks matched the predictive performance of established summary feature-based approaches and identified clock regions where spatial organization differed by cognitive status. After external validation, this approach has the potential to be incorporated within a clinical workflow to support automated and interpretable cognitive screening.

Objective: Case-finding models for underdiagnosed conditions are usually trained on diagnosis codes, which are missing from the records of many patients with a clinical diagnosis. We asked whether models trained on self-reported diagnoses rank uncoded self-reporters higher than models trained on later codes, and how a score fit only from released summary counts compares with a participant-level model. Methods: In the All of Us Research Program, we followed survey respondents with no qualifying diagnosis code by the survey date for later codes. For chronic fatigue, fibromyalgia, and endometriosis (participants recorded as female at birth), we fit a naive Bayes score with empirical-Bayes shrinkage, from released feature-by-label counts for the self-reported diagnosis and from exact counts inside the data enclave for a later code, and an elastic net on participant-level data for each label. In a random 20% validation split, we measured discrimination of self-report, of uncoded self-reporters (no qualifying code observed), and of later codes. Results: For chronic fatigue (149,860 respondents, including 7,770 self-reporters, 95% with no code observed), the summary-count score trained on self-report ranked uncoded self-reporters with an area under the receiver operating characteristic curve (AUROC) of 0.74 (95% CI, 0.72-0.75), against 0.67 (95% CI, 0.65-0.69) when trained on later codes. For later codes, the two scores were similar (0.69 vs 0.70). Fibromyalgia showed the same pattern (0.73 vs 0.64 and 0.74 vs 0.72). Elastic nets trained on self-report ranked uncoded self-reporters higher than the summary-count scores (0.80 vs 0.74 and 0.78 vs 0.73) but discriminated later codes less well than code-trained elastic nets. For endometriosis, the self-report score also ranked uncoded self-reporters above the code-trained score (0.61 vs 0.53), but discrimination was low and it did not discriminate later codes (0.50 vs 0.70). Conclusion: The choice of training label changes which patients a case-finding model ranks highly. For all 3 conditions, self-reported diagnoses were the more useful label for finding uncoded self-reporters. For chronic fatigue and fibromyalgia, the self-report score also discriminated later codes as well as the code-trained score. Independent clinical validation is needed before these models guide case finding.

Estimating the correlation between subject-specific rates of change is a common objective in longitudinal studies. A natural approach is to fit ordinary least squares regressions per subject and correlate the resulting slope estimates directly. Under complete data, this slopes-first estimator is algebraically equivalent to a contrast-based plug-in estimator derived from the joint variance-covariance matrix of the two signals across time. Under independent dropout, however, the slopes-first approach implicitly applies heterogeneous weights across missing-data patterns, introducing bias whenever the cross-covariance between the two signals is not constant over time. We formalise this result and propose an unbiased alternative that estimates the joint covariance matrix before applying the contrast. We further extend our results to include covariates as well. Simulations confirm that the slopes-first estimator is biased under non-constant cross-covariance while the proposed alternative remains unbiased. Further, we show in a real-world example that the two estimators can lead to different scientific conclusions and recommend that researchers working with longitudinal data inspect the missing data pattern to make an informed decision about the estimator of choice.

Large language models (LLMs) are increasingly evaluated for clinical tasks, but safe use requires reliable uncertainty estimates as well as high accuracy. We tested eight general-purpose LLMs on 100 validated neurology board-style multiple-choice questions and compared three confidence signals: self-reported confidence, critique-implied confidence from an independent model instance, and consistency-based confidence from repeated sampling. Accuracy varied from 45% (Llama 3.1 8B) to 92% (Gemini 3 Flash/Pro), but confidence was generally overestimated, particularly in lower-accuracy models. Consistency-based confidence provided the best discrimination between correct and incorrect answers (pooled AUROC 0.74), outperforming self-report, although calibration differences between methods were not significant. Calibration should therefore be reported alongside accuracy when clinical LLMs are evaluated.

Purpose To evaluate the external transportability of source-derived probability calibration for chest radiograph deep learning classifiers under distribution shift and determine its dependence on sample size and disease prevalence. Materials and Methods This retrospective secondary computational analysis of public, de-identified radiographic datasets involved no participant contact or new data collection. Cohort sizes were prespecified by availability without prospective power calculation: 106,686 NIH ChestX-ray14 images (29,323 patients; 1992-2015) for development and internal testing, 20,000 CheXpert images (20,000 patients; October 2002-July 2017), and 14,998 VinBigData images (2018-2020) for external testing. ResNet-18, DenseNet-121, and DeiT-Tiny classifiers for cardiomegaly and pleural effusion were trained across four patient fractions and three seeds (36 fits). Calibrators were fitted exclusively on NIH data. Twelve primary external Brier-score contrasts (temperature-scaled versus raw probabilities) were evaluated using paired cluster-bootstrapping with Holm adjustment. Results Temperature scaling reduced Brier scores in nine of 12 primary contrasts but worsened pleural effusion calibration on VinBigData across all architectures (Brier increases: ResNet-18, 0.001460 [95% CI: 0.001347, 0.001568]; DenseNet-121, 0.001193 [95% CI: 0.001109, 0.001274]; DeiT-Tiny, 0.000498 [95% CI: 0.000417, 0.000577]; all adjusted P=.006). Standardizing class weights to NIH prevalence reversed all CheXpert improvements. Although full versus 10% training improved external AUROC across all comparisons, DeiT-Tiny CheXpert pleural effusion Brier score worsened by 0.007039 despite a 0.032466 AUROC gain. At 80% selective image retention, calibrated ensembles retained only 12.3%-17.2% of VinBigData positive pleural effusion examinations. Conclusion Source-fitted calibration is not uniformly transportable across health systems, and discrimination gains do not guarantee calibrated probabilities. External evaluation must simultaneously assess probability reliability, prevalence shifts, and retained-case composition.